Scrambled packet payload mapping for robust transmission of data

ABSTRACT

Systems and methods for transmitting data partitioned into a sequence of frames may include transmitting a first packet that includes a primary frame and one or more preceding frames from the sequence of frames of data, wherein the one or more preceding frames of the first packet are separated from the primary frame of the first packet in the sequence of frames by respective multiples of a stride parameter; transmitting a second packet that includes a primary frame and one or more preceding frames from the sequence of frames of data, wherein the primary frame of the first packet is one of the one or more preceding frames of the second packet; and, prior to transmitting the first packet and the second packet, randomly determining an order of transmission for the first packet and the second packet.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of U.S. ProvisionalApplication Patent Ser. No. 63/175,352 filed Apr. 15, 2021, the entiredisclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

This disclosure relates to scrambled packet payload mapping for robusttransmission of data.

BACKGROUND

When working with data transmission over network applications there aretwo main types of Internet Protocol (IP) traffic, Transmission ControlProtocol (TCP) and User Datagram Protocol (UDP). Each protocol has itsown intended use and strengths.

For example, TCP is what's commonly known as an acknowledged modeprotocol. This means that when data is being exchanged, a built-infeedback mechanism checks and acknowledges whether the data was receivedcorrectly. If any data is missing or lost, further mechanisms retransmitthe corrupt or missing information. These mechanisms make TCPparticularly well suited for transferring information such as stillimages, data files and web pages. While this reliability makes TCP wellsuited for those use cases, it does come at a cost. In order toguarantee stability, each client receives their own TCP stream (known asunicasting) which means if there are many clients on the network, datais often replicated. These control and feedback mechanisms result in alarger protocol overhead, which means that a larger percentage of thevaluable bandwidth on a network connection is being used for sendingthis additional control information. Aside from additional bandwidthoverhead, the retransmission of data can cause an increase in latency,making real-time applications suffer.

UDP, on the other hand, is an unacknowledged mode protocol, meaning thatthere is no active mechanism to retransmit data that has been lostduring the exchange. As such, in certain scenarios UDP would be a badway to send and receive data compared to TCP—for example, sending anemail or downloading a file. However, other factors make UDP superiorfor real-time communications. With UDP being unacknowledged, there arefewer protocol overheads to take up valuable space that could be usedfor useful data, making transmission quicker and more efficient.

SUMMARY

Disclosed herein are implementations of scrambled packet payload mappingfor robust transmission of data.

In first aspect, a system is provided for transmitting data partitionedinto a sequence of frames of data. The system includes a memory, aprocessor, and a network interface. The memory stores instructionsexecutable by the processor to cause the system to: transmit, using thenetwork interface, a first packet that includes a primary frame and twoor more preceding frames from the sequence of frames of data, wherein atleast one of the two or more preceding frames of the first packet isseparated from the primary frame of the first packet in the sequence offrames by a respective multiple of a first stride parameter and at leastone of the two or more preceding frames of the first packet is separatedfrom the primary frame of the first packet in the sequence of frames bya respective multiple of a second stride parameter that is differentfrom the first stride parameter; transmit, using the network interface,a second packet that includes a primary frame and two or more precedingframes from the sequence of frames of data, wherein at least one of thetwo or more preceding frames of the second packet is separated from theprimary frame of the second packet in the sequence of frames by arespective multiple of the first stride parameter and at least one ofthe two or more preceding frames of the second packet is separated fromthe primary frame of the second packet in the sequence of frames by arespective multiple of the second stride parameter, and wherein theprimary frame of the first packet is one of the two or more precedingframes of the second packet; and, prior to transmitting the first packetand the second packet, randomly determine an order of transmission forthe first packet and the second packet.

In a second aspect, a method is provided for transmitting datapartitioned into a sequence of frames of data. The method includes:transmitting, using a network interface, a first packet that includes aprimary frame and two or more preceding frames from the sequence offrames of data, wherein at least one of the two or more preceding framesof the first packet is separated from the primary frame of the firstpacket in the sequence of frames by a respective multiple of a firststride parameter and at least one of the two or more preceding frames ofthe first packet is separated from the primary frame of the first packetin the sequence of frames by a respective multiple of a second strideparameter that is different from the first stride parameter;transmitting, using the network interface, a second packet that includesa primary frame and two or more preceding frames from the sequence offrames of data, wherein at least one of the two or more preceding framesof the second packet is separated from the primary frame of the secondpacket in the sequence of frames by a respective multiple of the firststride parameter and at least one of the two or more preceding frames ofthe second packet is separated from the primary frame of the secondpacket in the sequence of frames by a respective multiple of the secondstride parameter, and wherein the primary frame of the first packet isone of the two or more preceding frames of the second packet; and, priorto transmitting the first packet and the second packet, randomlydetermining an order of transmission for the first packet and the secondpacket.

In a third aspect, a non-transitory computer-readable storage medium isprovided for transmitting data partitioned into a sequence of frames ofdata. The non-transitory computer-readable storage medium includesexecutable instructions that, when executed by a processor, facilitateperformance of operations, including: transmitting, using a networkinterface, a first packet that includes a primary frame and two or morepreceding frames from the sequence of frames of data, wherein at leastone of the two or more preceding frames of the first packet is separatedfrom the primary frame of the first packet in the sequence of frames bya respective multiple of a first stride parameter and at least one ofthe two or more preceding frames of the first packet is separated fromthe primary frame of the first packet in the sequence of frames by arespective multiple of a second stride parameter that is different fromthe first stride parameter; transmitting, using the network interface, asecond packet that includes a primary frame and two or more precedingframes from the sequence of frames of data, wherein at least one of thetwo or more preceding frames of the second packet is separated from theprimary frame of the second packet in the sequence of frames by arespective multiple of the first stride parameter and at least one ofthe two or more preceding frames of the second packet is separated fromthe primary frame of the second packet in the sequence of frames by arespective multiple of the second stride parameter, and wherein theprimary frame of the first packet is one of the two or more precedingframes of the second packet; and, prior to transmitting the first packetand the second packet, randomly determining an order of transmission forthe first packet and the second packet.

In fourth aspect, a system is provided for transmitting data partitionedinto a sequence of frames of data. The system includes a memory, aprocessor, and a network interface. The memory stores instructionsexecutable by the processor to cause the system to: transmit, using thenetwork interface, a first packet that includes a primary frame and oneor more preceding frames from the sequence of frames of data, whereinthe one or more preceding frames of the first packet are separated fromthe primary frame of the first packet in the sequence of frames byrespective multiples of a stride parameter; transmit, using the networkinterface, a second packet that includes a primary frame and one or morepreceding frames from the sequence of frames of data, wherein the one ormore preceding frames of the second packet are separated from theprimary frame of the second packet in the sequence of frames byrespective multiples of the stride parameter, and wherein the primaryframe of the first packet is one of the one or more preceding frames ofthe second packet; and, prior to transmitting the first packet and thesecond packet, randomly determine an order of transmission for the firstpacket and the second packet.

In a fifth aspect, a method is provided for transmitting datapartitioned into a sequence of frames of data. The method includes:transmitting a first packet that includes a primary frame and one ormore preceding frames from the sequence of frames of data, wherein theone or more preceding frames of the first packet are separated from theprimary frame of the first packet in the sequence of frames byrespective multiples of a stride parameter; transmitting a second packetthat includes a primary frame and one or more preceding frames from thesequence of frames of data, wherein the one or more preceding frames ofthe second packet are separated from the primary frame of the secondpacket in the sequence of frames by respective multiples of the strideparameter, and wherein the primary frame of the first packet is one ofthe one or more preceding frames of the second packet; and, prior totransmitting the first packet and the second packet, randomlydetermining an order of transmission for the first packet and the secondpacket.

In a sixth aspect, a non-transitory computer-readable storage medium isprovided for transmitting data partitioned into a sequence of frames ofdata. The non-transitory computer-readable storage medium includesexecutable instructions that, when executed by a processor, facilitateperformance of operations, including: transmitting a first packet thatincludes a primary frame and one or more preceding frames from thesequence of frames of data, wherein the one or more preceding frames ofthe first packet are separated from the primary frame of the firstpacket in the sequence of frames by respective multiples of a strideparameter; transmitting a second packet that includes a primary frameand one or more preceding frames from the sequence of frames of data,wherein the one or more preceding frames of the second packet areseparated from the primary frame of the second packet in the sequence offrames by respective multiples of the stride parameter, and wherein theprimary frame of the first packet is one of the one or more precedingframes of the second packet; and, prior to transmitting the first packetand the second packet, randomly determining an order of transmission forthe first packet and the second packet.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is best understood from the following detaileddescription when read in conjunction with the accompanying drawings. Itis emphasized that, according to common practice, the various featuresof the drawings are not to-scale. On the contrary, the dimensions of thevarious features are arbitrarily expanded or reduced for clarity.

FIG. 1 is a block diagram of an example of a system for transferringframed data from a server to one or more client devices using packetpayload mapping for robust transmission of data.

FIG. 2 is a timing diagram of an example of packet encoding scheme usingpacket payload mapping for robust transmission of data.

FIGS. 3A and 3B are memory maps of an example of a ring buffer withentries that each hold the primary frame and the one or more precedingframes of a packet.

FIG. 4 is a block diagram of an example of an internal configuration ofa computing device of the system shown in FIG. 1.

FIG. 5 is a flowchart illustrating an example of a process fortransmitting data partitioned into a sequence of frames of data usingpacket payload mapping for robust transmission of data.

FIG. 6 is a flowchart illustrating an example of a process for receivingdata partitioned into a sequence of frames of data using packet payloadmapping for robust transmission of data.

FIG. 7 is a flowchart illustrating an example of a process for readingframes of data from a buffer with entries that each hold a primary frameand the one or more preceding frames of a packet.

FIG. 8 is a block diagram of an example of a data flow in a computingdevice configured to receive data partitioned into a sequence of framesof data using packet payload mapping for robust transmission of data.

FIG. 9 is a flowchart illustrating an example of a process for passingframed data through a reassembly buffer after removing packet payloadredundancy.

FIG. 10 is a flowchart illustrating an example of a process fortransmitting data partitioned into a sequence of frames of data usingpacket payload mapping with multiple strides for robust transmission ofdata.

FIG. 11 is a flowchart illustrating an example of a process forreceiving data partitioned into a sequence of frames of data usingpacket payload mapping with multiple strides for robust transmission ofdata.

FIG. 12A is a flowchart illustrating an example of a process forrandomizing a transmission order for packets bearing data partitionedinto a sequence of frames of data.

FIG. 12B is a flowchart illustrating an example of a process forrandomizing a transmission order for packets bearing data partitionedinto a sequence of frames of data.

FIG. 13 shows four bar graphs illustrating burst packet loss with andwithout sender-side scrambling of an order of transmission.

FIG. 14 shows two bar graphs illustrating burst packet loss with andwithout sender-side scrambling of an order of transmission.

DETAILED DESCRIPTION

When viewed through the lens of real-time data transmission, UDP becomesthe preferred transport layer protocol due to the lack of overhead andor low latency constraints. However, the lack of retransmission posesthe problem of stability and reliability. In general, it is acceptedthat some loss will occur when using UDP (hence the crackles that appearin audio or the glitches that appear in video). The packet-frame mappingschemes described herein may provide methods to recover some of thislost data.

Packet-frame mapping provides a way to distribute redundant data toallow for recovery data loss when using UDP. Other methods exist toachieve this type of recovery (such as Forward Error Correction), butpacket-frame mapping may use different ways of distributing and orderingits redundant data that may provide advantages over prior systems. Forexample, systems implementing packet-frame mapping for transmission offramed data may be more robust to long bursts of packet loss, which maybe common in some network environments, such as congested high-trafficwireless networks.

There are a couple terms to define when describing a packet-framemapping scheme. As used herein, “stride” is a spacing in terms of numberof frames in a sequence of frames of data between frames that areselected for inclusion in a given packet. For example, the stride of apacket-frame mapping scheme may correspond to the minimum number ofsequential packets that a client device is confident in receiving afterlosing a maps worth of data (e.g., the longest packet loss burst thescheme is designed to fully recover from. This number may be chosenbased on received packet statistics.

As used herein, “segment” is a redundancy parameter indicating thenumber of redundant frames being bundle into a packet.

As used herein, the “length” of a packet-frame map is the lowest frameindex or largest absolute index, relative to the current frame with anindex of zero, in the map. For example, in an audio stream, the lengthmultiplied by the frame size, is how much audio can be recovered. Forexample, for a packet-frame map with a length of 64 and a frame size of2.5 ms, 160 ms of audio can be recovered using this map.

As used herein, a packet map data structure indicates the stride and thesegment of the packet mapping scheme and may specify the structure ofthe packet payloads encoded on the network or other communicationchannel. For example, a server device may use the packet map datastructure to create and send the data to decode packet payloads andinstantiate buffers for receiving the packets to exploit the redundancy.For example, {0, −16, −32, −48, −64} is a map with a stride of 16 and asegment of 4. The packet frame mapping may be designed to recover dataafter high packet loss events. It can also deal with occasional packetloss; however, it shines in large loss events. For example, during ascanning event, an iOS device could lose 45 to 55 packets and thenreceive 16 packets, before losing 45 to 55 packets again. In thisscenario, a stride of 16 and a segment of 4 would allow for the recoveryof burst packet losses up to 64 packets long, and thus make the systemrobust to the iOS scanning events.

A packet frame mapping with this style of packet-frame map isparticularly good at recovery for a stable, recurring loss pattern. Forexample, when a wireless scan happens on a mobile device, the wirelesscard leaves its primary channel and hops to other channels to do quickscans, searching to see if there is a more performant channel to switchto. As a result, a pattern may occur that looks like this: 50 packetslost, 20 packets received, 50 packets lost, 20 packets received, 50packets lost, 20 packets received, and so on. For context, this is dueto the wireless card hopping off the primary channel, scanning, andhopping back. Hence the loss, receive, loss pattern. The examplepacket-frame map allows full recovery in this scenario.

As powerful as a packet-frame map with a single stride is, there arelimitations to this pattern-based recovery. For example, a stride of 16indicates that the devices needs to receive 16 packets after a lossevent to guarantee full recovery from the largest supported packet lossevent. In noisy wireless environments, it may be common to lose packetsevery couple of received packets in a random, pattern free fashion. Thistype of environment paired with a wireless scan would cause asingle-stride packet-frame mapping to fail unless it dramaticallyreduced its stride. For example, a single-stride map with a stride of 4and a length of 64: {0, −4, −8, −12, −16, −20, −24, −28, −32, −36, −40,−44, −48, −52, −56, −60, −64} has to use a large segment of 16, whichincreases the. As illustrated, this increases the number of frames inthe packet and as a result increases network bandwidth consumption ofthe packet-frame map scheme.

To better handle occasional or sporadic packet loss, a packet-frame mapmay be expanded to support multiple strides. For example, a short stride(e.g., 1 or 2) may be paired with a longer stride (e.g., 8 or 16) tohandle packet loss with different profiles. Each stride in apacket-frame map may have a respective segment that specifies the numberof redundant frames per packet at that stride.

In some implementations, by varying the stride a multi-stridepacket-frame map can be used, which can help address both packet lossscenarios nicely. For example, the packet-frame map {0, −1, −2, −3, −4,−8, −16, −32, −48, −64} has a stride of 1 with segment of 4 and a strideof 8 with a segment of 5 and would be able handle small frequent packetloss events thanks to the small stride at the beginning of the map,while also being able to recover larger amounts of data thanks to thelarge length value. The total number of redundant frames per packet isthe sum of the segments across all the strides in the packet-frame map,which in this example is 4+5=9 redundant frames per packet.

A system for implementing a packet-frame mapping scheme, may include aserver to package and send the data according to a packet map datastructure as well as a client which can unpack and utilize the redundantdata correctly. In some implementations, a client device uses twoseparate types of buffers: a ring buffer and a reassembly buffer. Thesize of the ring buffer may be determined by the size of the stride andthe segment. In our example, the stride is 16 and the segment is 4, sothe ring buffer may be sized to store frames from (stride×segment)+1packets=65 packets. In some implementations, the ring buffer may besized to store frames from (stride×segment) packets=64 packets and alatest packet or live packet, that bears as its oldest frame a copy of aframe that is the newest frame of the oldest packet stored in the ringbuffer, may be stored outside of the ring buffer until after the oldestpacket has been deleted from the ring buffer. For example, if we haveframes that are audio data corresponding to audio signal of duration 2.5milliseconds and the buffers are sized to recover up to 64 frames (e.g.,the lowest number in the packet map data structure) the latencyintroduced by the buffering scheme would be 2.5 milliseconds*64=160milliseconds. For example, the size of the reassembly buffer may bedependent on the size of the stride, in this case 16. The purpose of thering buffer is to receive and figure out which data has been lost. Thenthe data may be reconstructed in the reassembly buffer in order allowingfor the lost data to be recovered and played correctly.

Some implementations may employ sender-side scrambling, i.e.,randomizing a transmission order for packets bearing the framed data inaccordance with a packet mapping scheme. Instead of only creating abuffer client side and having the client store and then reorder theframes, a buffer may be added on the server side and the order oftransmission for the packets with redundancy may be randomly scrambledbefore transmitting them to the client. While the order of primaryframes is scrambled, the redundant frames are still paired with theirsame respective primary frames in accordance with a packet mappingscheme. Sender-side scrambling of the transmission order may provideadvantages, such as, increasing resilience to burst packet loss events,making the system more robust.

FIG. 1 is a block diagram of an example of a system 100 for transferringframed data from a server to one or more client devices using packetpayload mapping for robust transmission of data. The system 100 includesa server device 102 with a network interface 104 configured to send andreceive data via a network 106. The network 106 can include, forexample, the Internet, and/or the network 106 can be, or include, alocal area network (LAN), a wide area network (WAN), a virtual privatenetwork (VPN), or any other public or private means of electroniccomputer communication capable of transferring data between computingdevices (e.g., a client and a server). The network 106 or any otherelement, or combination of elements, of the system 100 can includenetwork hardware such as routers, switches, load balancers, othernetwork devices, or combinations thereof. For example, the network 106may be a WiFi network. For example, the network 106 may be a cellulartelephony network. For example, the server device 102 may include anaudio server that transmits audio data to client devices via the network106. In some implementations, the server device 102 is configured tomulticast audio data of a live concert or musical performance topersonal computing devices (e.g., smartphones or tablets) of concertgoers via wireless network (e.g., a WiFi network) in a stadium or othervenue. For example, the server device 102 may include a video serverthat transmits video data to client devices via the network 106. Forexample, the server device 102 may include a file server that transmitsfiles to client devices via the network 106. For example, the serverdevice may include components of the computing device 400 of FIG. 4. Forexample, the server device 102 may implement the process 500 of FIG. 5.For example, the server device 102 may implement the process 1000 ofFIG. 10.

The system 100 also includes a client device 110 and a client device112. The client device 110 includes a network interface 120 configuredto send and receive data via a network 106. The client device 112includes a network interface 122 configured to send and receive data viaa network 106. The client device 110 includes a ring buffer 130 that maybe used facilitate the reception of framed data using packet payloadmapping for robust transmission of data. The ring buffer 130 may allowfor the use redundancy in a packet payload mapping to recover from lossof packets during transmission, including potentially long bursts ofpacket loss. For example, the ring buffer 130 may include entries thateach hold the primary frame and the one or more preceding frames of apacket. In some implementations, the ring buffer 130 may store multiplecopies of frames of data and be configured to output a single copy ofeach frame in a sequence of frames of data. For example, the ring buffer130 may be provisioned based on one or more parameters of a packetpayload mapping scheme, such as a stride parameter and/or a redundancyparameter. For example, the ring buffer 130 may be the ring buffer 302of FIGS. 3A and 3B. Similarly, the client device 112 includes a ringbuffer 132 that may be used facilitate the reception of framed datausing packet payload mapping for robust transmission of data. Forexample, the ring buffer 132 may be the ring buffer 302 of FIGS. 3A and3B. For example, the client device 110 may include components of thecomputing device 400 of FIG. 4. For example, the client device 112 mayinclude components of the computing device 400 of FIG. 4. For example,the client device 110 and the client device 112 may implement theprocess 600 of FIG. 6. For example, the client device 110 and the clientdevice 112 may implement the process 1100 of FIG. 11.

The server device 102 is configured to transmit packets, including apacket 140, that include one or more redundant frames of data spaced ina sequence of frames by a number of frames determined by a strideparameter for a packet payload mapping scheme. For example, in a packetpayload mapping scheme where a stride parameter is five and a redundancyparameter is three, the packet 140 may include a primary frame withframe index I, a preceding frame with frame index 1-5, a preceding framewith frame index I-10, and a preceding frame with frame index 1-15. Forexample, the packet encoding scheme 200 of FIG. 2 may be used to encodethe packet 140 and other packets transmitted by the server device 102.In some implementations, the packet 140 is multicast by the serverdevice 102 to the client device 110 and the client device 112. In someimplementations, the packet 140 is broadcast by the server device 102 toall devices on the network 106. In some implementations, the packet 140is unicast by the server device 102 to client device 110. Note, twoclient devices 110 and 112 are shown in FIG. 1 for simplicity, but manymore client devices could be supported by the server device 102.

FIG. 2 is a timing diagram of an example of packet encoding scheme 200using packet payload mapping for robust transmission of data. The packetencoding scheme 200 uses a stride parameter of five, which means theframes included in a packet are spaced in a sequence of frames of databy five frames from each other. The packet encoding scheme 200 usesredundancy parameter of three, which means the packets include a primary(e.g., most recent) frame and three preceding frames, which may havebeen transmitted in previous packets as a primary frame and/or as apreceding frame, spaced according to the stride parameter.

The first packet transmitted is packet zero 210, which includes aprimary frame 212 with a frame index of 0, a preceding frame 214 with aframe index of −5 (i.e., five frames before frame 0), a preceding frame216 with a frame index of −10, and a preceding frame 218 with a frameindex of −15. The second packet transmitted is packet one 220, whichincludes a primary frame 222 with a frame index of 1, a preceding frame224 with a frame index of −4, a preceding frame 226 with a frame indexof −9, and a preceding frame 228 with a frame index of −14. The thirdpacket transmitted is packet two 230, which includes a primary frame 232with a frame index of 2, a preceding frame 234 with a frame index of −3,a preceding frame 236 with a frame index of −8, and a preceding frame238 with a frame index of −13. The fourth packet transmitted is packetthree 240, which includes a primary frame 242 with a frame index of 3, apreceding frame 244 with a frame index of −2, a preceding frame 246 witha frame index of −7, and a preceding frame 248 with a frame index of−12. The fifth packet transmitted is packet four 250, which includes aprimary frame 252 with a frame index of 4, a preceding frame 254 with aframe index of −1, a preceding frame 256 with a frame index of −6, and apreceding frame 258 with a frame index of −11.

For example, the packet encoding scheme 200 enables to recovery of aburst of packet loss of up to 15 packets (i.e., stride times redundancy)based on the reception of five consecutive packets (i.e., strideconsecutive packets). For example, the packet encoding scheme 200 may bewell suited to network environments that are prone to long bursts ofpacket loss.

FIG. 3A shows a memory map 300 of an example of a ring buffer 302 withentries 310-340 that each hold the primary frame and the one or morepreceding frames of a packet. In this example, the ring buffer 302 issized to support a packet payload mapping scheme (e.g., the packetencoding scheme 200 of FIG. 2) with a stride parameter of five and aredundancy parameter of three. The outer ring of the memory map 300represents the primary frames stored for each packet. The three innerrings of the memory map 300 represents the preceding frames stored foreach packet. In the memory map 300, the entry 310 of the ring buffer 302corresponds to the oldest packet with frames currently stored in thering buffer 302. The entry 340 of the ring buffer 302 corresponds to thenewest packet with frames currently stored in the ring buffer 302. Theentry 310 is next entry to be evaluated to read the next frame of datafrom the ring buffer 302. Since the entry 310 stores a valid copy of theframe with index 0 as its primary frame (i.e., the corresponding packetwas successfully received), the frame with index 0 may be read as theprimary frame from entry 310 of the ring buffer 302. Responsive to theprimary frame of the entry 310 being read from the ring buffer 302, theentry 310 may be deleted (e.g., the memory locations may be released forreuse). Note that entry 340 includes a copy of the frame with index 0 inits oldest (innermost) preceding frame slot. By waiting until the packetcorresponding to entry 340 is received to read the entry 310 with thisframe as its primary frame, the chance of having a valid copy of theframe with index zero at the time of the read from the ring buffer 302is maximized. Thus, the ring buffer 302 may be sized to store framesfrom a number (i.e., 16 in this example) of packets equal to the(stride×redundancy)+1. For real-time media transmissions (e.g., audio orvideo), this buffering scheme may introduce an algorithmic latencycorresponding to the frame duration (e.g., 2.5 milliseconds or 5milliseconds) times the stride parameter times the redundancy parameter.

For example, the ring buffer 302 may be implemented a circular buffer.In some implementations, the ring buffer 302 may take input from packetsoutput from a jitter buffer and the ring buffer 302 may operate as afirst-in-first-out (FIFO) buffer.

FIG. 3B shows a memory map 350 of the ring buffer 302 with entries thateach hold the primary frame and the one or more preceding frames of apacket. The memory map 350 shows the ring buffer 302 at a time laterthan the state in the memory map 300 of FIG. 3A. The entry 310 has beenupdated to store to the frames of a newest packet that has beenreceived. This newest packet includes a primary frame with frame index16. Now, the entry 312 corresponds to the oldest packet that should haveits frames stored in the ring buffer 302. However, the packetcorresponding to the entry 312 has been lost (e.g., corrupted orotherwise lost). Therefore, the next frame on the sequence of frameswith index 1 is not available as the primary frame in entry 312. Insteadthe ring buffer 302 may be searched for a valid copy of this next framefor read out. In this example, there are two valid copies of the framewith index 1 stored as preceding frames in the ring buffer in entry 332and in entry 310. One of these copies of the frame with index 1 may thenbe read as a preceding frame from the ring buffer 302.

FIG. 4 is a block diagram of an example of an internal configuration ofa computing device 400 of the system shown in FIG. 1, such as a serverdevice 102, the client device 110, and/or the client device 112. Forexample, a client device and/or a server device can be a computingsystem including multiple computing devices and/or a single computingdevice, such as a mobile phone, a tablet computer, a laptop computer, anotebook computer, a desktop computer, a server computer, and/or othersuitable computing devices. A computing device 400 can includecomponents and/or units, such as a processor 402, a bus 404, a memory406, peripherals 414, a power source 416, a network communication unit418, a user interface 420, other suitable components, and/or anycombination thereof.

The processor 402 can be a central processing unit (CPU), such as amicroprocessor, and can include single or multiple processors, havingsingle or multiple processing cores. Alternatively, the processor 402can include another type of device, or multiple devices, now existing orhereafter developed, capable of manipulating or processing information.For example, the processor 402 can include multiple processorsinterconnected in any manner, including hardwired and/or networked,including wirelessly networked. In some implementations, the operationsof the processor 402 can be distributed across multiple physical devicesand/or units that can be coupled directly or across a local area orother type of network. In some implementations, the processor 402 caninclude a cache, or cache memory, for local storage of operating dataand/or instructions. The operations of the processor 402 can bedistributed across multiple machines, which can be coupled directly oracross a local area or other type of network.

The memory 406 can include volatile memory, non-volatile memory, and/ora combination thereof. For example, the memory 406 can include volatilememory, such as one or more DRAM modules such as DDR SDRAM, andnon-volatile memory, such as a disk drive, a solid state drive, flashmemory, Phase-Change Memory (PCM), and/or any form of non-volatilememory capable of persistent electronic information storage, such as inthe absence of an active power supply. The memory 406 can includeanother type of device, or multiple devices, now existing or hereafterdeveloped, capable of storing data and/or instructions for processing bythe processor 402. The processor 402 can access and/or manipulate datain the memory 406 via the bus 404. Although shown as a single block inFIG. 4, the memory 406 can be implemented as multiple units. Forexample, a computing device 400 can include volatile memory, such asRAM, and persistent memory, such as a hard drive or other storage. Thememory 406 can be distributed across multiple machines, such asnetwork-based memory or memory in multiple machines performing theoperations of clients and/or servers.

The memory 406 can include executable instructions 408; data, such asapplication data 410; an operating system 412; or a combination thereoffor immediate access by the processor 402. The executable instructions408 can include, for example, one or more application programs, whichcan be loaded and/or copied, in whole or in part, from non-volatilememory to volatile memory to be executed by the processor 402. Theexecutable instructions 408 can be organized into programmable modulesand/or algorithms, functional programs, codes, code segments, and/orcombinations thereof to perform various functions described herein. Forexample, the memory 406 may include instructions executable by theprocessor 402 to cause a system including the computing device 400 toimplement the process 500 of FIG. 5, the process 600 of FIG. 6, theprocess 1000 of FIG. 10, or the process 1100 of FIG. 11.

The application data 410 can include, for example, user files; databasecatalogs and/or dictionaries; configuration information for functionalprograms, such as a web browser, a web server, a database server; and/ora combination thereof. The operating system 412 can be, for example,Microsoft Windows®, Mac OS X®, or Linux®; an operating system for asmall device, such as a smartphone or tablet device; or an operatingsystem for a large device, such as a mainframe computer. The memory 406can comprise one or more devices and can utilize one or more types ofstorage, such as solid state or magnetic storage.

The peripherals 414 can be coupled to the processor 402 via the bus 404.The peripherals can be sensors or detectors, or devices containing anynumber of sensors or detectors, which can monitor the computing device400 itself and/or the environment around the computing device 400. Forexample, a computing device 400 can contain a geospatial locationidentification unit, such as a global positioning system (GPS) locationunit. As another example, a computing device 400 can contain atemperature sensor for measuring temperatures of components of thecomputing device 400, such as the processor 402. Other sensors ordetectors can be used with the computing device 400, as can becontemplated. In some implementations, a client and/or server can omitthe peripherals 414. In some implementations, the power source 416 canbe a battery, and the computing device 400 can operate independently ofan external power distribution system. Any of the components of thecomputing device 400, such as the peripherals 414 or the power source416, can communicate with the processor 402 via the bus 404. Althoughdepicted here as a single bus, the bus 404 can be composed of multiplebuses, which can be connected to one another through various bridges,controllers, and/or adapters.

The network communication unit 418 can also be coupled to the processor402 via the bus 404. In some implementations, the network communicationunit 418 can comprise one or more transceivers. The networkcommunication unit 418 provides a connection or link to a network, suchas the network 106, via a network interface, which can be a wirednetwork interface, such as Ethernet, or a wireless network interface.For example, the computing device 400 can communicate with other devicesvia the network communication unit 418 and the network interface usingone or more network protocols, such as Ethernet, TCP, IP, power linecommunication (PLC), WiFi, infrared, GPRS, GSM, CDMA, TDMA, UMTS, orother suitable protocols.

A user interface 420 can include a display; a positional input device,such as a mouse, touchpad, touchscreen, or the like; a keyboard; and/orany other human and machine interface devices. The user interface 420can be coupled to the processor 402 via the bus 404. Other interfacedevices that permit a user to program or otherwise use the computingdevice 400 can be provided in addition to or as an alternative to adisplay. In some implementations, the user interface 420 can include adisplay, which can be a liquid crystal display (LCD), a cathode-ray tube(CRT), a light emitting diode (LED) display (e.g., an OLED display), orother suitable display. The user interface 420 may include an audiodriver (e.g., a speaker) configured to convert electronic audio data tosound in medium (e.g., air). For example, a speaker of the userinterface 420 may be used to play audio data (e.g., encoding music orspeech signals).

FIG. 5 is a flowchart illustrating an example of a process 500 fortransmitting data partitioned into a sequence of frames of data usingpacket payload mapping for robust transmission of data. The process 500includes transmitting 502 a packet map data structure that indicates astride parameter and a redundancy parameter; for a sequence of frames ofdata, selecting 510 a next frame from the sequence as a primary framefor transmission, and transmitting 520 a packet including a primaryframe and one or more preceding frames at a spacing in the sequence offrames of data corresponding to the stride parameter; when (at operation525) the last frame has been transmitted 520 as a primary frame,transmitting 530 packets including a dummy primary frame correspondingto a next sequence index past the last frame of the sequence along withone or more preceding frames at the stride spacing from the dummy index,until (at operation 535) the last frame of the sequence of frames hasbeen transmitted a number of times equal to one plus the redundancyparameter; and ending 540 the data transmission. For example, theprocess 500 of FIG. 5 may be implemented by the server device 102 ofFIG. 1. For example, the process 500 of FIG. 5 may be implemented by thecomputing device 400 of FIG. 4.

The process 500 includes transmitting 502 a packet map data structurethat indicates the stride parameter and indicates a count of redundantpreceding frames to be transmitted in each packet of a set of packetsthat include frames from a sequence of frames of data. The strideparameter may specify a spacing in the sequence of frames between aprimary packet and one or more preceding packets included in a packet ofthe packets. In some implementations, the stride parameter is greaterthan four. The count of redundant preceding frames may be referred to asthe redundancy parameter for a packet payload mapping scheme. Forexample, the packet map data structure may include a list of integersthat are frame index offsets of corresponding preceding frames, with oneinteger for each preceding frame (e.g., {−5, −10, −15} for a scheme withstride 5 and redundancy 3 or {−16, −32, −48, −64} for a scheme withstride 16 and redundancy 4). In some implementations, the packet mapdata structure includes two positive integers representing the strideparameter and the redundancy parameter respectively. The packet map datastructure may be transmitted 502 using a network interface (e.g., thenetwork interface 104). In some implementations, the packet map datastructure is transmitted 502 once in a separate packet before the framesof data start to be transmitted 520. In some implementations, the packetmap data structure is transmitted 502 as part of a header for each ofthe packets in the set of packets used to transmit the sequence offrames. For example, the sequence of frames of data may be a sequence offrames of audio data (e.g., encoding music or speech signals). Forexample, the sequence of frames of data may be a sequence of frames ofvideo data.

The process 500 includes selecting 510 a next frame as a primary framefor transmission in the next packet. For example, a next frame pointeror counter may be initialized to point to a frame at the beginning ofthe sequence of frames of data, and the frame pointer or counter may beincremented or otherwise updated to select 510 the next frame fortransmission as a primary frame. In some implementations, an order oftransmission for packets bearing the frames of data may be randomized tomitigate the effects of large burst packet loss events on the packetmapping scheme. For example, the process 500 may include, prior totransmitting a first packet and a second packet, randomly determining anorder of transmission for the first packet and the second packet. Framesof data for packets to be transmitted may be buffered to enableout-of-frame-order transmission. Various sizes for the send-side buffermay be used, depending on the level of burst loss tolerance desired andthe tolerance for end-to-end latency in an application. For example, asender-side buffer may be sized to store data for a number of packetsmatching a number of packets stored by a ring buffer for the packetmapping scheme in a corresponding receiver device. In someimplementations, randomly determining the order of transmission includesrandomly generating an index to a buffer with entries that store dataassociated with respective packets, including the first packet and thesecond packet, to select a next packet for transmission. A packet may begenerated based on the frames of data related to its primary frame bythe packet mapping scheme encoded by the packet map data structureeither before the random order for transmission is determined or afterthe random order for transmission is determined. For example, selecting510 a next frame as a primary frame for transmission in the next packetmay include implementing the process 1200 of FIG. 12A to randomize thetransmission order. For example, selecting 510 a next frame as a primaryframe for transmission in the next packet may include implementing theprocess 1250 of FIG. 12B to randomize the transmission order.

The process 500 includes transmitting 520 a packet that includes aprimary frame (e.g., the currently selected 510 frame) and one or morepreceding frames from the sequence of frames of data. The one or morepreceding frames of the first packet are separated from the primaryframe of the first packet in the sequence of frames by respectivemultiples of the stride parameter (e.g., as illustrated for the packetencoding scheme 200 for a stride parameter of 5). For example, thepacket may be transmitted 520 using a network interface (e.g., thenetwork interface 104). In some implementations, the one or morepreceding frames of the packet include two or more preceding frames fromthe sequence of frames of data. The packet may include other data, suchas header data (e.g., an IPv6 header and a UDP header) to facilitatetransmission of the packet across a network (e.g., the network 106). If(at operation 525) there are more frames in the sequence of frames ofdata to transmit, then the next frame may be selected 510 andtransmitted 520 as a primary frame in another packet, along with one ormore corresponding preceding frames with respect to the new primaryframe. When the process 500 is starting up, one or more of the precedingframes may be omitted from the packet where there is no correspondingearlier frame in the sequence of frames to re-transmit. After a numberof packets corresponding to the stride parameter have been transmitted,a frame that has previously been transmitted in an earlier packet (e.g.,a first packet) as a primary frame may be retransmitted as precedingframe in a later packet (e.g., a second packet). For example, theprocess 500 may include transmitting 520, using the network interface, asecond packet that includes a primary frame and one or more precedingframes from the sequence of frames of data, wherein the one or morepreceding frames of the second packet are separated from the primaryframe of the second packet in the sequence of frames by respectivemultiples of the stride parameter, and wherein the primary frame of thefirst packet is one of the one or more preceding frames of the secondpacket. In some implementations, the process 500 includes transmitting520, using the network interface, all frames between the primary frameof the first packet and the primary frame of the second packet in thesequence of frames of data as primary frames of respective packets thateach include one or more preceding frames from the sequence of frames ofdata separated in the sequence of frames by multiples of the strideparameter. The packets may be transmitted 520 via the network to one ormore client devices. For example, the first packet and the second packetmay be broadcast packets. For example, the first packet and the secondpacket may be multicast packets. In some implementations, frames in thesequence of frames of data are all a same size.

When (at operation 525) all of the frames of the sequence of frames havebeen transmitted 520 as primary frames of respective packets, one ormore packets including a dummy primary frame (or omitting a primaryframe) corresponding to a next sequence index past the last frame of thesequence with corresponding preceding frames at the stride may betransmitted 530, until (at operation 535) all frames have beentransmitted a number of times as preceding frames corresponding to theredundancy parameter. When (at operation 535) all the frames of thesequence have been transmitted the redundancy parameter times aspreceding frames, then the transmission of the sequence of frames ofdata ends 540.

FIG. 6 is a flowchart illustrating an example of a process 600 forreceiving data partitioned into a sequence of frames of data usingpacket payload mapping for robust transmission of data. The process 600includes receiving 610 a packet map data structure that indicates astride parameter and a redundancy parameter; receiving 620 packets thateach respectively include a primary frame and one or more precedingframes from the sequence of frames of data in accordance with a strideparameter; storing 630 the frames of the packets in a buffer withentries that each hold the primary frame and the one or more precedingframes of a packet; and reading 640 frames from the buffer to obtain asingle copy of the frames at the output of the buffer. For example, theprocess 600 of FIG. 6 may be implemented by the client device 110 ofFIG. 1. For example, the process 600 of FIG. 6 may be implemented by thecomputing device 400 of FIG. 4.

The process 600 includes receiving 610, using the network interface, apacket map data structure that indicates the stride parameter andindicates a count of redundant preceding frames to be transmitted ineach of the packets. The stride parameter may specify a spacing in thesequence of frames between a primary packet and one or more precedingpackets included in a packet of the packets. In some implementations,the stride parameter is greater than four. The count of redundantpreceding frames may be referred to as the redundancy parameter for apacket payload mapping scheme. For example, the packet map datastructure may include a list of integers that are frame index offsets ofcorresponding preceding frames, with one integer for each precedingframe (e.g., {−5, −10, −15} for a scheme with stride 5 and redundancy 3or {−16, −32, −48, −64} for a scheme with stride 16 and redundancy 4).In some implementations, the packet map data structure includes twopositive integers representing the stride parameter and the redundancyparameter respectively. For example, the packet map data structure maybe received 610 using a network interface (e.g., the network interface120). In some implementations, the packet map data structure is received610 once in a separate packet before the frames of data start to bereceived 620. In some implementations, the packet map data structure isreceived 610 as part of a header for each of the packets in the set ofpackets used to transmit the sequence of frames. For example, thesequence of frames of data may be a sequence of frames of audio data(e.g., encoding music or speech signals). For example, the sequence offrames of data may be a sequence of frames of video data.

The process 600 includes receiving 620 packets that each respectivelyinclude a primary frame and one or more preceding frames from thesequence of frames of data. The one or more preceding frames of arespective packet are separated from the primary frame of the respectivepacket in the sequence of frames by a respective multiple of the strideparameter (e.g., as illustrated for the packet encoding scheme 200 for astride parameter of 5). For example, the packets may be received 620using a network interface (e.g., the network interface 120). In someimplementations, the one or more preceding frames of each respectivepacket include a number of preceding frames equal to the redundancyparameter, and the redundancy parameter is greater than one. The packetsmay be received 620 via a network from one or more server devices (e.g.,the server device 102).

The process 600 includes storing 630 the frames of the packets in abuffer (e.g., the ring buffer 302) with entries that each hold theprimary frame and the one or more preceding frames of a packet. In someimplementations, the one or more preceding frames of each respectivepacket include a number of preceding frames equal to a redundancyparameter, and wherein the buffer is sized to store frames from a numberof packets equal to the stride parameter times the redundancy parameterplus one. In some implementations, the received 620 packets may bepassed through a jitter buffer before the frames from the payload of thepackets are stored 630 in the buffer. For example, the process 600 mayinclude storing the packets in a jitter buffer as they are received 620;and reading the packets from the jitter buffer before storing 630 theframes of the packets in the buffer. In some implementations, frames inthe sequence of frames of data are all a same size.

The process 600 includes reading 640 frames from the buffer (e.g., thering buffer 302). For example, the process 700 of FIG. 7 may beimplemented to read 640 frames from the buffer. In some implementations,the frames of data from the sequence include audio data, and the process600 includes playing audio data of a frame responsive to reading theframe from the buffer. In some implementations, frames of data read 640from the buffer are reassembled in the sequence of frames of data in areassembly buffer (e.g., the reassembly buffer 820 of FIG. 8). Forexample, the process 900 of FIG. 9 may be implemented to reassemble asequence of frames of data to facilitating playing audio data.

FIG. 7 is a flowchart illustrating an example of a process 700 forreading frames of data from a buffer with entries that each hold aprimary frame and the one or more preceding frames of a packet. Theprocess 700 includes determining 710, based on the buffer, whether apacket with a primary frame that is a next frame in the sequence offrames of data has been lost; when (at operation 715) the packet has notbeen lost, reading 720 the frame from the buffer as the primary framefrom one of the entries of the buffer; when (at operation 715) thepacket has been lost, finding 740 a redundant copy of the next frame inthe buffer as a preceding frame of a newer entry in the buffer, andreading 750 the next frame from the buffer as a preceding frame of anentry; and when (at operation 755) there are no more frames in thesequence of frames to read, ending 760 data reception. For example, theprocess 700 of FIG. 7 may be implemented by the client device 110 ofFIG. 1. For example, the process 700 of FIG. 7 may be implemented by thecomputing device 400 of FIG. 4.

The process 700 includes determining 710, based on the buffer (e.g., thering buffer 302), that a packet with a primary frame that is a nextframe in the sequence of frames of data has been lost. For example, theoldest entry in the buffer may be checked to determine 710 whether thepacket corresponding to the oldest entry was lost. In someimplementations, the entry includes either a pointer to valid data froma received packet or a null pointer if the corresponding packet waslost. In some implementations, the buffer includes an indication (e.g.,a flag) of whether the entry is currently storing valid frames of data,which may be set when data from a received packet is stored in the entryand reset when the entry is deleted 730 (e.g., after a read of theprimary frame). In some implementations, determining 710 a packet hasbeen lost includes performing a data integrity check (e.g., a cyclicredundancy check (CRC)) on data stored in an entry corresponding topacket of interest.

The process 700 includes, when (at operation 715) the packet has notbeen lost, reading 720 the frame (e.g., a first frame) from the bufferas the primary frame from one of the entries of the buffer. The process700 includes, responsive to the primary frame of an entry (e.g., a firstentry) of the buffer being read 720 from the buffer, deleting 730 theentry from the buffer. For example, the entry may be deleted 730 byreleasing the memory of the entry for reuse (e.g., to store frames of anewly received packet).

The process 700 includes, when (at operation 715) the packet has beenlost, finding 740 a redundant copy of the next frame in the buffer as apreceding frame of a newer entry in the buffer. For example, there maybe up to redundancy parameter copies of the frame stored in the bufferas preceding frames of later packets (e.g., as illustrated by the innerrings of the ring buffer 302 and as discussed in relation to FIGS.3A-B). One or more of these entries corresponding to packets bearing theframe as a preceding frame under the packet payload mapping scheme maybe checked to find 740 a valid copy of the frame. The process 700includes, responsive to the determination that the packet was lost,reading 750 the next frame from the buffer as a preceding frame from oneof the entries of the buffer. Although not explicitly shown in FIG. 7the process 700 may handle cases where none of the copies of the framehave been successfully received and no copy is found 740 in the bufferby returning an error message and/or interpolating the missing databased on earlier and/or later data in the sequence of frames of data.

When (at operation 755) there are no more frames in the sequence offrames of data to be read, data reception may be ended 760.

FIG. 8 is a block diagram of an example of a data flow 800 in acomputing device configured to receive data partitioned into a sequenceof frames of data using packet payload mapping for robust transmissionof data. A packet source 802 outputs packets that each respectivelyinclude a primary frame and one or more preceding frames from thesequence of frames of data. The one or more preceding frames of arespective packet are separated from the primary frame of the respectivepacket in the sequence of frames by a respective multiple of the strideparameter (e.g., as illustrated for the packet encoding scheme 200 for astride parameter of 5). For example, the packet source may be a networkcommunications unit (e.g., the network communications unit 418) thatreceives the packets via a network (e.g., the network 106) from a serverdevice (e.g., the server device 102). In some implementations, thepacket source 802 includes a jitter buffer that is configured to outputthe packets in transmitted order.

The data flow 800 includes a buffer (e.g., the ring buffer 302) withentries that each hold the primary frame and the one or more precedingframes of a packet. The preceding frames may be redundant copies offrame sent in earlier packets as primary frames. Thus, the buffer 810may store multiple copies of frames in the sequence of frames of data(e.g., as discussed in relation to FIGS. 3A-B). This redundancy may makethe data flow 800 robust to substantial bursts of packet loss in thepacket source 802.

The data flow 800 includes a reassembly buffer 820 with entries thateach hold a frame of the sequence of frames of data. Single copies ofthe frames may be read from the buffer 810 and written to the reassemblybuffer 820 to remove the redundancy that was used for transmission onthe noisy network and/or to properly order the frames, thus reassemblingthe frames into a useable form of the source data (e.g., reassembling afile that partitioned into frames or queuing a larger super frame ofaudio or video data for play out in a user interface). For example, thereassembly buffer 820 may have a size corresponding to a number offrames equal to the stride parameter. In some implementations that use amulti-stride packet-frame map, the reassembly buffer 820 may be sized tostore a number of frames equal to a largest stride parameter of apacket-frame map used by the received packets. For example, thereassembly buffer 820 may be a first-in-first-out (FIFO) buffer. Forexample, the reassembly buffer 820 may be implemented as a circularbuffer.

The data flow 800 includes a data sink 822 that consumes the framed datafrom the reassembly buffer 820. For example, the data sink 822 mayinclude an audio driver (e.g., including a speaker) that is used to playaudio data in the frames. For example, the data sink 822 may be file innon-volatile memory that is being written to with the framed data.

FIG. 9 is a flowchart illustrating an example of a process 900 forpassing framed data through a reassembly buffer after removing packetpayload redundancy. The process 900 includes playing 910 audio data thatis stored in an oldest frame from the reassembly buffer; deleting 920the oldest frame from the reassembly buffer; and writing 930 frames thatare read from the buffer to a reassembly buffer. For example, theprocess 900 of FIG. 9 may be implemented by the client device 110 ofFIG. 1. For example, the process 900 of FIG. 9 may be implemented by thecomputing device 400 of FIG. 4.

The process 900 includes playing 910 audio data that is stored in anoldest frame from a reassembly buffer (e.g., the reassembly buffer 820).For example, the audio data may be played 910 using an audio driver(e.g., a speaker) of the user interface 420. The process 900 includesdeleting 920 the oldest frame from the reassembly buffer. For example,the entry may be deleted 920 by releasing the memory of the entry forreuse (e.g., by updating a pointer or counter of the reassembly buffer).

The process 900 includes writing frames (e.g., including a first frameand a next frame) that are read from a buffer (e.g., the buffer 810) toa reassembly buffer (e.g., the reassembly buffer). The reassembly buffermay include entries that each consist of a frame of the sequence offrames of data. In some implementations, the reassembly buffer is sizedto store a number of frames equal to a stride parameter.

FIG. 10 is a flowchart illustrating an example of a process 1000 fortransmitting data partitioned into a sequence of frames of data usingpacket payload mapping with multiple strides for robust transmission ofdata. The process 1000 includes transmitting 1002 a packet map datastructure that indicates multiple stride parameters and correspondingredundancy parameters (e.g., strides and corresponding segments); for asequence of frames of data, selecting 1010 a next frame from thesequence as a primary frame for transmission, and transmitting 1020 apacket including a primary frame and two or more preceding frames atspacings in the sequence of frames of data corresponding to the multiplestride parameters; when (at operation 1025) the last frame has beentransmitted 1020 as a primary frame, transmitting 1030 packets includinga dummy primary frame corresponding to a next sequence index past thelast frame of the sequence along with one or more preceding frames atthe stride spacings from the dummy index, until (at operation 1035) thelast frame of the sequence of frames has been transmitted a number oftimes equal to one plus the sum of the redundancy parameters for themultiple stride parameters; and ending 1040 the data transmission. Forexample, the process 1000 of FIG. 10 may be implemented by the serverdevice 102 of FIG. 1. For example, the process 1000 of FIG. 10 may beimplemented by the computing device 400 of FIG. 4.

The process 1000 includes transmitting 1002, a packet map data structurethat indicates multiple stride parameters, including a first strideparameter and a second stride parameter, and indicates counts ofredundant preceding frames for each stride parameter to be transmittedin each packet of a set of packets that include frames from a sequenceof frames of data. Each stride parameter may specify a spacing in thesequence of frames between a primary packet and one or more precedingpackets included in a packet of the packets. In some implementations,the first stride parameter is less than three and the second strideparameter is greater than four. The counts of redundant preceding framesmay be referred to as the redundancy parameters for respective strideparameters for a packet payload mapping scheme. For example, the packetmap data structure may include a list of integers that are frame indexoffsets of corresponding preceding frames, with one integer for eachpreceding frame (e.g., {−1, −2, −5, −10, −15} for a scheme with a firststride of 1 with a corresponding redundancy of 2 and a second stride of5 with a corresponding redundancy of 3 or {−2, −4, −6, −16, −32, −48,−64} for a scheme with a first stride of 2 with a correspondingredundancy of 3 and a second stride of 16 with a correspondingredundancy of 4). In some implementations, the packet map data structureincludes two positive integers for each stride parameter representing astride parameter and its corresponding redundancy parameterrespectively. The packet map data structure may be transmitted 1002using a network interface (e.g., the network interface 104). In someimplementations, the packet map data structure is transmitted 1002 oncein a separate packet before the frames of data start to be transmitted1020. In some implementations, the packet map data structure istransmitted 1002 as part of a header for each of the packets in the setof packets used to transmit the sequence of frames. For example, thesequence of frames of data may be a sequence of frames of audio data(e.g., encoding music or speech signals). For example, the sequence offrames of data may be a sequence of frames of video data.

The process 1000 includes selecting 1010 a next frame as a primary framefor transmission in the next packet. For example, a next frame pointeror counter may be initialized to point to a frame at the beginning ofthe sequence of frames of data, and the frame pointer or counter may beincremented or otherwise updated to select 1010 the next frame fortransmission as a primary frame. In some implementations, an order oftransmission for packets bearing the frames of data may be randomized tomitigate the effects of large burst packet loss events on the packetmapping scheme. For example, the process 1000 may include, prior totransmitting a first packet and a second packet, randomly determining anorder of transmission for the first packet and the second packet. Framesof data for packets to be transmitted may be buffered to enableout-of-frame-order transmission. Various sizes for the send-side buffermay be used, depending on the level of burst loss tolerance desired andthe tolerance for end-to-end latency in an application. For example, asender-side buffer may be sized to store data for a number of packetsmatching a number of packets stored by a ring buffer for the packetmapping scheme in a corresponding receiver device. In someimplementations, randomly determining the order of transmission includesrandomly generating an index to a buffer with entries that store dataassociated with respective packets, including the first packet and thesecond packet, to select a next packet for transmission. A packet may begenerated based on the frames of data related to its primary frame bythe packet mapping scheme encoded by the packet map data structureeither before the random order for transmission is determined or afterthe random order for transmission is determined. For example, selecting1010 a next frame as a primary frame for transmission in the next packetmay include implementing the process 1200 of FIG. 12A to randomize thetransmission order. For example, selecting 1010 a next frame as aprimary frame for transmission in the next packet may includeimplementing the process 1250 of FIG. 12B to randomize the transmissionorder.

The process 1000 includes transmitting 1020 a packet (e.g., a firstpacket) that includes a primary frame (e.g., the currently selected 1010frame) and two or more preceding frames from the sequence of frames ofdata. The two or more preceding frames may be selected from the sequenceof frames of data for inclusion in the packet based on a multi-stridepacket-frame map specified by the packet map data structure. In someimplementations, at least one of the two or more preceding frames of thepacket is separated from the primary frame of the packet in the sequenceof frames by a respective multiple of a first stride parameter and atleast one of the two or more preceding frames of the packet is separatedfrom the primary frame of the packet in the sequence of frames by arespective multiple of a second stride parameter that is different fromthe first stride parameter. For example, the packet may be transmitted1020 using a network interface (e.g., the network interface 104). Insome implementations, more than two strides may be used in each packet.For example, at least one of the two or more preceding frames of thepacket may be separated from the primary frame of the packet in thesequence of frames by a respective multiple of a third stride parameterthat is different from the first stride parameter and the second strideparameter. The packet may include other data, such as header data (e.g.,an IPv6 header and a UDP header) to facilitate transmission of thepacket across a network (e.g., the network 106).

If (at operation 1025) there are more frames in the sequence of framesof data to transmit, then the next frame may be selected 1010 andtransmitted 1020 as a primary frame in another packet, along with one ormore corresponding preceding frames with respect to the new primaryframe. When the process 1000 is starting up, one or more of thepreceding frames may be omitted from the packet where there is nocorresponding earlier frame in the sequence of frames to re-transmit.After a number of packets corresponding to the stride parameter havebeen transmitted, a frame that has previously been transmitted in anearlier packet (e.g., a first packet) as a primary frame may beretransmitted as preceding frame in a later packet (e.g., a secondpacket). For example, the process 1000 may include transmitting 1020,using the network interface, a second packet that includes a primaryframe and two or more preceding frames from the sequence of frames ofdata, wherein at least one of the two or more preceding frames of thesecond packet is separated from the primary frame of the second packetin the sequence of frames by a respective multiple of the first strideparameter and at least one of the two or more preceding frames of thesecond packet is separated from the primary frame of the second packetin the sequence of frames by a respective multiple of the second strideparameter, and wherein the primary frame of the first packet is one ofthe two or more preceding frames of the second packet. In someimplementations, the process 1000 includes transmitting 1020, using thenetwork interface, all frames between the primary frame of the firstpacket and the primary frame of the second packet in the sequence offrames of data as primary frames of respective packets that each includetwo or more preceding frames from the sequence of frames of dataseparated in the sequence of frames by multiples of the first strideparameter and the second stride parameter. The packets may betransmitted 1020 via the network to one or more client devices. Forexample, the first packet and the second packet may be broadcastpackets. For example, the first packet and the second packet may bemulticast packets. In some implementations, frames in the sequence offrames of data are all a same size.

When (at operation 1025) all of the frames of the sequence of frameshave been transmitted 1020 as primary frames of respective packets, oneor more packets including a dummy primary frame (or omitting a primaryframe) corresponding to a next sequence index past the last frame of thesequence with corresponding preceding frames at the stride may betransmitted 1030, until (at operation 1035) all frames have beentransmitted a number of times as preceding frames corresponding to thetotal of the redundancy parameters (e.g., segments for respectivestrides) of the packet map data structure. When (at operation 1035) allthe frames of the sequence have been transmitted the in every redundancyslot of the packet-frame map, then the transmission of the sequence offrames of data ends 1040.

FIG. 11 is a flowchart illustrating an example of a process 1100 forreceiving data partitioned into a sequence of frames of data usingpacket payload mapping with multiple strides for robust transmission ofdata. The process 1100 includes receiving 1110 a packet map datastructure that indicates multiple stride parameters and correspondingredundancy parameters (e.g., strides and corresponding segments);receiving 1120 packets that each respectively include a primary frameand two or more preceding frames from the sequence of frames of data atspacings in the sequence of frames of data corresponding to the multiplestride parameters; storing 1130 the frames of the packets in a bufferwith entries that each hold the primary frame and the two or morepreceding frames of a packet; and reading 1140 frames from the buffer toobtain a single copy of the frames at the output of the buffer. Forexample, the process 1100 of FIG. 11 may be implemented by the clientdevice 110 of FIG. 1. For example, the process 1100 of FIG. 11 may beimplemented by the computing device 400 of FIG. 4.

The process 1100 includes receiving 1110 a packet map data structurethat indicates multiple stride parameters, including a first strideparameter and a second stride parameter, and indicates counts ofredundant preceding frames for each stride parameter to be transmittedin each packet of a set of packets that include frames from a sequenceof frames of data. Each stride parameter may specify a spacing in thesequence of frames between a primary packet and one or more precedingpackets included in a packet of the packets. In some implementations,the first stride parameter is less than three and the second strideparameter is greater than four. The counts of redundant preceding framesmay be referred to as the redundancy parameters (e.g., segments) forrespective stride parameters for a packet payload mapping scheme. Forexample, the packet map data structure may include a list of integersthat are frame index offsets of corresponding preceding frames, with oneinteger for each preceding frame (e.g., {−1, −2, −5, −10, −15} for ascheme with a first stride of 1 with a corresponding redundancy of 2 anda second stride of 5 with a corresponding redundancy of 3 or {−2, −4,−6, −16, −32, −48, −64} for a scheme with a first stride of 2 with acorresponding redundancy of 3 and a second stride of 16 with acorresponding redundancy of 4). In some implementations, the packet mapdata structure includes two positive integers for each stride parameterrepresenting a stride parameter and its corresponding redundancyparameter respectively. For example, the packet map data structure maybe received 1110 using a network interface (e.g., the network interface120). In some implementations, the packet map data structure is received1110 once in a separate packet before the frames of data start to bereceived 1120. In some implementations, the packet map data structure isreceived 1110 as part of a header for each of the packets in the set ofpackets used to transmit the sequence of frames. For example, thesequence of frames of data may be a sequence of frames of audio data(e.g., encoding music or speech signals). For example, the sequence offrames of data may be a sequence of frames of video data.

The process 1100 includes receiving 1120 packets that each respectivelyinclude a primary frame and two or more preceding frames from thesequence of frames of data. The two or more preceding frames may havebeen selected from the sequence of frames of data for inclusion in thepacket based on a multi-stride packet-frame map specified by the packetmap data structure. In some implementations, at least one of the two ormore preceding frames of a respective packet is separated from theprimary frame of the respective packet in the sequence of frames by arespective multiple of a first stride parameter and at least one of thetwo or more preceding frames of the respective packet is separated fromthe primary frame of the respective packet in the sequence of frames bya respective multiple of a second stride parameter that is differentfrom the first stride parameter. For example, the packets may bereceived 1120 using a network interface (e.g., the network interface120). In some implementations, the two or more preceding frames of eachrespective packet include a number of preceding frames equal to a sum ofredundancy parameters for respective stride parameters, and theredundancy parameter for the second stride parameter is greater thanone. The packets may be received 1120 via a network from one or moreserver devices (e.g., the server device 102).

The process 1100 includes storing 1130 the frames of the packets in abuffer (e.g., the ring buffer 302) with entries that each hold theprimary frame and the two or more preceding frames of a packet. In someimplementations, the two or more preceding frames of each respectivepacket include a number of preceding frames equal to a sum of redundancyparameters for respective stride parameters, including the first strideparameter and the second stride parameter. For example, the buffer maybe sized to store frames from a number of packets equal to a largeststride parameter times the respective redundancy parameter for thelargest stride parameter plus one. In some implementations, the received1120 packets may be passed through a jitter buffer before the framesfrom the payload of the packets are stored 1130 in the buffer. Forexample, the process 1100 may include storing the packets in a jitterbuffer as they are received 1120; and reading the packets from thejitter buffer before storing 1130 the frames of the packets in thebuffer. In some implementations, frames in the sequence of frames ofdata are all a same size.

The process 1100 includes reading 1140 frames from the buffer (e.g., thering buffer 302). For example, the process 700 of FIG. 7 may beimplemented to read 1140 frames from the buffer. In someimplementations, the frames of data from the sequence include audiodata, and the process 1100 includes playing audio data of a frameresponsive to reading the frame from the buffer. In someimplementations, frames of data read 1140 from the buffer arereassembled in the sequence of frames of data in a reassembly buffer(e.g., the reassembly buffer 820 of FIG. 8). For example, the process900 of FIG. 9 may be implemented to reassemble a sequence of frames ofdata to facilitating playing audio data.

FIG. 12A is a flowchart illustrating an example of a process 1200 forrandomizing a transmission order for packets bearing data partitionedinto a sequence of frames of data. The process 1200 includes storing1210 packets, including a first packet and a second packet, in a bufferto await transmission in a randomly determined order. The packetpayloads, including redundant frames of data, may be stored with orwithout various levels of communication protocol headers. Various sizesfor the send-side buffer may be used, depending on the level of burstloss tolerance desired and the tolerance for end-to-end latency in anapplication. For example, a sender-side buffer may be sized to storedata for a number of packets matching a number of packets stored by aring buffer for the packet mapping scheme in a corresponding receiverdevice.

The process 1200 includes, randomly determining 1220 an order oftransmission for the first packet and the second packet. In someimplementations, randomly determining 1220 the order of transmissionincludes randomly generating an index to a buffer with entries thatstore data associated with respective packets, including the firstpacket and the second packet, to select a next packet for transmission.For example, the index to the buffer may be determined using a randomnumber generator (RNG) of an operating system (e.g., the Linux kernelRNG). For example, the process 1200 of FIG. 12A may be implemented bythe server device 102 of FIG. 1. For example, the process 1200 of FIG.12A may be implemented by the computing device 400 of FIG. 4.

FIG. 12B is a flowchart illustrating an example of a process 1250 forrandomizing a transmission order for packets bearing data partitionedinto a sequence of frames of data. In this example, packets aregenerated 1280 after the primary frame has been identified as next in arandomized order of transmission. The process 1250 may conserve memoryby avoiding buffering redundant copies of frames of data that arepresent in the assembled packets under the packet mapping scheme. Theprocess 1250 includes storing 1260 frames, including the primary frameof a first packet and the primary frame of a second packet, in a bufferto await transmission in the randomly determined order. Various sizesfor the send-side buffer may be used, depending on the level of burstloss tolerance desired and the tolerance for end-to-end latency in anapplication. For example, a sender-side buffer may be sized to storedata for a number of packets matching a number of packets stored by aring buffer for the packet mapping scheme in a corresponding receiverdevice.

The process 1250 includes randomly determining 1270 an order oftransmission for a first packet and a second packet. In someimplementations, randomly determining 1270 the order of transmissionincludes randomly generating an index to a buffer with entries thatstore data associated with respective packets, including the firstpacket and the second packet, to select a next packet for transmission.For example, the index to the buffer may be determined using a randomnumber generator (RNG) of an operating system (e.g., the Linux kernelRNG).

The process 1250 includes, responsive to the primary frame of a firstpacket becoming the next frame in the randomly determined order fortransmission as a primary frame, generating the first packet. The firstpacket may be generated based on the frames of data related to itsprimary frame by the packet mapping scheme encoded by the packet mapdata structure. For example, the process 1250 of FIG. 12B may beimplemented by the server device 102 of FIG. 1. For example, the process1250 of FIG. 12B may be implemented by the computing device 400 of FIG.4.

FIG. 13 shows four bar graphs illustrating burst packet loss with andwithout sender-side scrambling of an order of transmission. In these bargraphs (1310, 1320, 1330, and 1340) the height of the line represents alive data frame sequence number. The bar graph 1310 illustrates datatransmission without sender-side scrambling using a fixed order oftransmission for the packets. The bar graph 1320 illustrates receptionof the same packets subject to a burst loss pattern.

The bar graph 1330 illustrates data transmission with sender-sidescrambling that randomly determines an order of transmission for thepackets. The bar graph 1340 illustrates reception of the same packetstransmitted in the randomized order subject to the same burst losspattern shown in bar graph 1320. Note that the same burst loss patternis now spread across the larger transmission.

FIG. 14 shows two bar graphs illustrating burst packet loss with andwithout sender-side scrambling of an order of transmission. FIG. 14presents another view of burst loss amortized over a larger transmissionwindow. In these bar graphs (1410 and 1420) the x axis corresponds toframe number of the primary frames and the y axis is binary. Bars thatare present/non-zero height correspond to packets that have beensuccessfully received. Bars that are absent/zero height correspond topackets that have been lost received. The bar graph 1410 illustratesreception of a sequence of packets that have been transmitted withoutscrambling. The bar graph 1420 illustrates reception of a sequence ofpackets that have been transmitted with scrambling. Scrambling, with arandomly determined order of transmission, may provide advantages, suchas, by amortizing loss over a longer period of time, the actualconsecutive loss amount becomes smaller which may both lends itself topacket map recovery as well as enable better data interpolation for lostframe data.

While the disclosure has been described in connection with certainembodiments, it is to be understood that the disclosure is not to belimited to the disclosed embodiments but, on the contrary, is intendedto cover various modifications and equivalent arrangements includedwithin the scope of the appended claims, which scope is to be accordedthe broadest interpretation so as to encompass all such modificationsand equivalent structures as is permitted under the law.

What is claimed is:
 1. A system for transmitting data partitioned into asequence of frames of data, comprising: a network interface; a memory;and a processor, wherein the memory includes instructions executable bythe processor to cause the system to: transmit, using the networkinterface, a first packet that includes a primary frame and two or morepreceding frames from the sequence of frames of data, wherein at leastone of the two or more preceding frames of the first packet is separatedfrom the primary frame of the first packet in the sequence of frames bya respective multiple of a first stride parameter and at least one ofthe two or more preceding frames of the first packet is separated fromthe primary frame of the first packet in the sequence of frames by arespective multiple of a second stride parameter that is different fromthe first stride parameter; transmit, using the network interface, asecond packet that includes a primary frame and two or more precedingframes from the sequence of frames of data, wherein at least one of thetwo or more preceding frames of the second packet is separated from theprimary frame of the second packet in the sequence of frames by arespective multiple of the first stride parameter and at least one ofthe two or more preceding frames of the second packet is separated fromthe primary frame of the second packet in the sequence of frames by arespective multiple of the second stride parameter, and wherein theprimary frame of the first packet is one of the two or more precedingframes of the second packet; and prior to transmitting the first packetand the second packet, randomly determine an order of transmission forthe first packet and the second packet.
 2. The system of claim 1,wherein the memory includes instructions executable by the processor tocause the system to: store packets, including the first packet and thesecond packet, in a buffer to await transmission in the randomlydetermined order.
 3. The system of claim 1, wherein the memory includesinstructions executable by the processor to cause the system to: storeframes, including the primary frame of the first packet and the primaryframe of the second packet, in a buffer to await transmission in therandomly determined order; and responsive to the primary frame of thefirst packet becoming the next frame in the randomly determined orderfor transmission as a primary frame, generate the first packet.
 4. Thesystem of claim 1, wherein the first stride parameter is less than threeand the second stride parameter is greater than four.
 5. The system ofclaim 1, wherein at least one of the two or more preceding frames of thefirst packet is separated from the primary frame of the first packet inthe sequence of frames by a respective multiple of a third strideparameter that is different from the first stride parameter and thesecond stride parameter.
 6. The system of claim 1, wherein the memoryincludes instructions executable by the processor to cause the systemto: transmit, using the network interface, all frames between theprimary frame of the first packet and the primary frame of the secondpacket in the sequence of frames of data as primary frames of respectivepackets that each include two or more preceding frames from the sequenceof frames of data separated in the sequence of frames by multiples ofthe first stride parameter and the second stride parameter.
 7. Thesystem of claim 1, wherein the memory includes instructions executableby the processor to cause the system to: transmit, using the networkinterface, a packet map data structure that indicates the first strideparameter and the second stride parameter and indicates counts ofredundant preceding frames for each stride parameter to be transmittedin each packet of a set of packets including the first packet and thesecond packet.
 8. A method for transmitting data partitioned into asequence of frames of data, comprising: transmitting, using a networkinterface, a first packet that includes a primary frame and two or morepreceding frames from the sequence of frames of data, wherein at leastone of the two or more preceding frames of the first packet is separatedfrom the primary frame of the first packet in the sequence of frames bya respective multiple of a first stride parameter and at least one ofthe two or more preceding frames of the first packet is separated fromthe primary frame of the first packet in the sequence of frames by arespective multiple of a second stride parameter that is different fromthe first stride parameter; transmitting, using the network interface, asecond packet that includes a primary frame and two or more precedingframes from the sequence of frames of data, wherein at least one of thetwo or more preceding frames of the second packet is separated from theprimary frame of the second packet in the sequence of frames by arespective multiple of the first stride parameter and at least one ofthe two or more preceding frames of the second packet is separated fromthe primary frame of the second packet in the sequence of frames by arespective multiple of the second stride parameter, and wherein theprimary frame of the first packet is one of the two or more precedingframes of the second packet; and prior to transmitting the first packetand the second packet, randomly determining an order of transmission forthe first packet and the second packet.
 9. The method of claim 8,wherein randomly determining the order of transmission comprises:randomly generating an index to a buffer with entries that store dataassociated with respective packets, including the first packet and thesecond packet, to select a next packet for transmission.
 10. The methodof claim 8, comprising: storing packets, including the first packet andthe second packet, in a buffer to await transmission in the randomlydetermined order.
 11. The method of claim 8, comprising: storing frames,including the primary frame of the first packet and the primary frame ofthe second packet, in a buffer to await transmission in the randomlydetermined order; and responsive to the primary frame of the firstpacket becoming the next frame in the randomly determined order fortransmission as a primary frame, generating the first packet.
 12. Themethod of claim 8, wherein the first stride parameter is less than threeand the second stride parameter is greater than four.
 13. The method ofclaim 8, comprising: transmitting, using the network interface, allframes between the primary frame of the first packet and the primaryframe of the second packet in the sequence of frames of data as primaryframes of respective packets that each include two or more precedingframes from the sequence of frames of data separated in the sequence offrames by multiples of the first stride parameter and the second strideparameter.
 14. The method of claim 8, comprising: transmitting, usingthe network interface, a packet map data structure that indicates thefirst stride parameter and the second stride parameter and indicatescounts of redundant preceding frames for each stride parameter to betransmitted in each packet of a set of packets including the firstpacket and the second packet.
 15. A system for transmitting datapartitioned into a sequence of frames of data, comprising: a networkinterface; a memory; and a processor, wherein the memory includesinstructions executable by the processor to cause the system to:transmit, using the network interface, a first packet that includes aprimary frame and one or more preceding frames from the sequence offrames of data, wherein the one or more preceding frames of the firstpacket are separated from the primary frame of the first packet in thesequence of frames by respective multiples of a stride parameter;transmit, using the network interface, a second packet that includes aprimary frame and one or more preceding frames from the sequence offrames of data, wherein the one or more preceding frames of the secondpacket are separated from the primary frame of the second packet in thesequence of frames by respective multiples of the stride parameter, andwherein the primary frame of the first packet is one of the one or morepreceding frames of the second packet; and prior to transmitting thefirst packet and the second packet, randomly determine an order oftransmission for the first packet and the second packet.
 16. The systemof claim 15, wherein the memory includes instructions executable by theprocessor to cause the system to: store packets, including the firstpacket and the second packet, in a buffer to await transmission in therandomly determined order.
 17. The system of claim 15, wherein thememory includes instructions executable by the processor to cause thesystem to: store frames, including the primary frame of the first packetand the primary frame of the second packet, in a buffer to awaittransmission in the randomly determined order; and responsive to theprimary frame of the first packet becoming the next frame in therandomly determined order for transmission as a primary frame, generatethe first packet.
 18. The system of claim 15, wherein the strideparameter is greater than four.
 19. The system of claim 15, wherein thememory includes instructions executable by the processor to cause thesystem to: transmit, using the network interface, all frames between theprimary frame of the first packet and the primary frame of the secondpacket in the sequence of frames of data as primary frames of respectivepackets that each include one or more preceding frames from the sequenceof frames of data separated in the sequence of frames by multiples ofthe stride parameter.
 20. The system of claim 15, wherein the memoryincludes instructions executable by the processor to cause the systemto: transmit, using the network interface, a packet map data structurethat indicates the stride parameter and indicates a count of redundantpreceding frames to be transmitted in each packet of a set of packetsincluding the first packet and the second packet.